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The Numbers 

• Nearly 1 .5 million students graduate each year 
having taken the SAT® 

• When students register to take the SAT, they 
are asked to fill out a 42-item questionnaire 

• In the mid-1990s, web registration for the SAT 
began 

• By 2002, 44% of the graduating cohort 
registered on the web 
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The Population & The Instrument 

• The SAT is primarily taken by juniors and seniors in 
high school 

• The SAT 1 is a test that measures critical reading, 
math, and writing 

• The SAT questionnaire includes questions about: 

• High school coursework and activities 

• College preferences and aspirations 

• Demographic information 


1 Writing was added in March 2005 
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Reports on the Cohort 

• Summary reports are produced based on the 
graduating class of seniors 

• If you last took the SAT as a junior (in January 2005) 
your scores will be part of a summary report on all 
seniors graduating in 2006 

• Issue-Time lag for results 

• Impact of questionnaire modification in 2003 has 
impact on about 75% students 
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Web increase — > Non-response increase 

• Key questions in which non-response caused the 
most concern: 

• Ethnicity 

• Mother's Education 

• Father's Education 

• Parental Income 

• High School GPA 

• First Language Spoken 

• Best Language Spoken 
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The Web Questionnaire in 2002-03 


describe Yourself 




|— Select Your Choice— _£] 

Unguiges 


What laiicjil.icje -cli-cl you teii 11 to speak fir Si? 

C English only C En^sh and anther hangup C Another language 

Wlw! lant|tiHkj« do ynu Know best? 

r English C Engisli and another language about the same C Anotlw language 


Key research questions were 

not required 



...and these questions were 
scattered across multiple 
screens 

▼ 

WTmUs your parents or guar titans' highest level of education? 


Educ^lioii 

Mother (or female guardian) 

Fdthei (or male guardian) 

Grade school 

r 

r 

Some high school 

r 

r 

High school djjfoma or eiftwatent 

r 

r 

Business Of Irade school 

r 

r 

Some to iege 

r 

r 

Associate or iwo-yesr decree 

r 

r 

Bechets or Tow-year degree 

r 

r 

Some ^*duae or proiesstohst school 

r 

r 

Graduate or proiesstohet degree 

r 

r 

1 <*om liftow 

r 

r 


Reset 

R£!£l 


Grade Paint Average 


Select your curnuteflive t&Jtt scfwoi average for all academic subjects , 

r a+ r a r a- r b* c b re. rc+ c c r c- c d+ c d ref 

97-100 93-36 90-92 37-59 33-36 30-32 70-79 73-76 70-72 67-69 65-66 Below 85 


wtuit is Hie jppioxirmte combined income of your parents before taxes ta&tyeu ? 


C u$$1hen$1O t 0CO 
r Abod$l 0.000 lo $15,000 
r Aboui $15.00010 mow 

r Atwdmoooio$Aw 

r Abttt^OOO lo $30,000 

r Atxjul $30,000 lo $S5,OOQ 
r Abort $35,000 lo $40,000 


C About $40,000 to $50,000 
r About $50,000 to $60,000 
r About $60,000 to $70,000 
r About $70,000 to $80,000 
C About $80,000 to $100,000 
C More than $100,000 
C | don't know. 


n 


CollegeBoard 


connect to college success™ 
www.collegeboard.com 








Web increase — ► Non-response increase 


Percent of Non-Response by Registration Type, 2002 



□ Paper 

□ Web 


Ethnicity Mother's Father's Parental 
Education Education Income 


HS GPA First Best 

Language Language 
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Actions Taken to Intervene 

• In 2002-03, a committee met to identify which 
questions are needed most for scoring, test 
validity, and equity 

• In May 2003, questions on the web were 
reordered, moving seven key questions to the 
beginning 
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Changes Applied to Questionnaire 

• Noted 'For research only' on parental questions 

• Removed option for students to 'Reset' or 
respond 'I don't know' 

• Added "I do not wish to respond" option 

• Removed 'Cancel' option to exit 

• Required all first-time web registrants to go to 
questionnaire 

• If respondent left 'key questions' blank, question 
reloads 
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A Current Look at the Web Questionnaire 


DESCRIBE YOURSELF 

Please choose a response from the menu below; 
1 1 do not wish to respond 

FOR RESEARCH ONLY 


Help 


? Help 


Your answers to the following three questions will be used for research purposes only* rn way? 
that ensure your privacy. They will not be sent to your high school or any colleges, 

Mother's (or female guardian's) highest level of education: 

| Some college 


iuired ) 


Father's (or male guardian's) highest level of education: 


| High school diploma or equivalent ^ U * Required "/ 

Approximate combined income Df your parents before taxes last year. Include taxable and 
ndntaxable income from all sources; 

(About $70,000 to $80,0-00 Required.. 


GRADE POINT AVERAGE 

Please select a grade point average , C* Required 


7 Help 


a 

A+ (57-100) 

r 

r 

A (33-96) 

r 

r 

A- (90-9Z) 

r 

r 

B+ (87-09) 

r 

r 

6 (S3-86) 

r 

r 

r 

B- (80-SE) 
C+ (77-79) 

r 


LANGUAGES 

What language did you learn to speak ft rsfc?< 
f *" English only 

^ English and another language 
Another language 
f I do not wish to respond 

What language do -you know best? 

English 

^ English and another language about the saint 
^ Another language 

I do not wish to respond 


• All key research 
questions are required 

• Language is included, 
to indicate that some 
of the questions are for 
'Research purposes 
only' 


• All questions are on 
the (same) first screen 
of the Web 
questionnaire 
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Two Years Ago We Committed To: 

• Review non-response percent in the 2005 cohort 

• Current effort underway to see if the question 
prompt is contributing to non-response in the 
ethnicity question 

• Explore ways to match students as they come to 
the website to answer the questionnaire a little at 
a time 
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Registration Patterns have changed 
dramatically over the last 5 years 


Percent by Registration Type 


100 



2002 2003 2004 2005 2006 


□ Paper 

□ Web 
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Non-Response in Key Questions 2 


Percent of Non-Response by Cohort Year 



□ 2002 

□ 2003 

□ 2004 

□ 2005 

□ 2006 


Education Education 


2 GPA moved from item #1 to #5 
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Non-Response in Key Questions (Web) 


Percent of Web Non-Response by Cohort Year 



Education Education 


□ 2002 

□ 2003 

□ 2004 

□ 2005 

□ 2006 
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Non-Response in Key Questions (Paper) 


Percent of Paper Non-Response by Cohort Year 



□ 2002 

□ 2003 

□ 2004 

□ 2005 

□ 2006 


Ethnicity Mother's Father's Parental Income HS GPA First Language Best Language 

Education Education 
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Non-Response in Questions that moved back 3 


Percent of Web Non-Response by Cohort Year 



□ 

□ 

□ 

□ 

□ 


2002 

2003 

2004 

2005 

2006 


3 Questions that moved to accommodate key questions; does not include 4 alternate major questions 
*GPA moved from item #1 to #5 
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What’s a Researcher To Do? 

• When you have so much to ask: 

• Identify most important items and ask them first 

• Identify core questions and link answers over time 
with unique identifier 

• Explore feasibility of adding 'I do not wish to 
respond' and/or reloading question once 
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Response Rate vs. Question Bias 


Percent of Students Noting Other to Ethnicity by 

Registration Type 



□ Overall 

□ Paper 

□ Web 
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Special Study to Examine Multi-Ethnic 

Response Option 

• In October 2005, the College Board conducted a 
study evaluating two different question formats 
enabling students to indicate multiple 
ethnicities. 

• Students nationwide participating in the October 
2005 SAT test administration had one of two 
ethnicity questions printed on the last page of 
their test booklet. 


^CollegeBoard 


connect to college success™ 
www.collegeboard.com 


Instructions appearing on the last page of 
the test booklet: 


The following questions are for research 
purposes only. Answering is voluntary 
and will not affect your scores in any 
way. The College Board is trying to 
determine how best to collect 
information about students to ensure 
that our tests and services are useful to 
all. Your responses will be kept 
confidential and will not be shared with 
anyone outside the College Board. 
Please mark your answers in the space 
provided on page 8 of your answer 
sheet. Thank you for your assistance. 


SPECIAL QUESTIONS 


1 ® ® © © © ®(§>®Q ® 

2 ® © © © © © © © CD (j : 

3 ® © © © © © © © 0 ® 

4 ® © © © © © © © 0 © 

5 © © © © © © © © 0 ® 

6 t* ;• © © © © © © © CD © : 

7 © © © © © © © © ® © 

8 ® © © © © © © © © ® 


(Back of Answer Booklet) 


& 
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2-Part Question 

1 . How do you describe yourself? (Select ALL that apply.) 

(A) American Indian or Alaska Native 

(B) Asian, Asian American, or Pacific Islander 

(C) Black or African American 

(D) Mexican or Mexican American 

(E) Puerto Rican 

(F) Other Hispanic, Latino, or Latin American 

(G) White 

(H) Other 

2. Which one of these groups would you say best describes your 

race/ethnicity? (Mark only one .) 

(A) American Indian or Alaska Native 

(B) ... 


^CollegeBoard 


connect to college success™ 
www.collegeboard.com 


3-Part Question 


1 . Which one of these groups would you say best describes your 

race/ethnicity? (Mark only one .) 

(A) American Indian or Alaska Native 

(B) Asian, Asian American, or Pacific Islander 

(C) Black or African American 

(D) Mexican or Mexican American 

(E) Puerto Rican 

(F) Other Hispanic, Latino, or Latin American 

(G) White 

(H) Other 

2. Are you multi-racial or multi-ethnic? 

(A) Yes 

(B) No 


3. If you answered yes to question 2, please select ALL of the groups below 
that describe your race/ethnicity. 

(A) American Indian or Alaska Native 

(B) ... 


^CollegeBoard 


22 


connect to college success™ 
www.collegeboard.com 


Test-Takers 


• Students nationwide participated in the study 
(N = 484,850). 

• 204,249 responses to 2-part question 

• 199,409 responses to 3-part question 

• However, non-response in the study was higher 
in both formats of the special study questions 
than non-response in the SAT Q (filled out by 
students prior to taking the test). 
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A Look at Non-response in the Study 


SAT Q 

2- part Special 
Study Question 

3- part Special 
Study Question 


Non-Response 

N 

40,367 

38,294 

42,898 


Non-Response 

% 

8.3 

15.8 

17.7 
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What did responses to the special study 
questions tell us about ethnic non-responders 
in the SAT Q? 

• A percentage of students who did not indicate an ethnicity 
prior to test-day did actually respond to the special study 
ethnicity questions. 

• Of those SAT Q Non-Respondents (n = 40,367) who 
participated in the study: 



N (%) indicating 

N (%) indicating 


ethnicity 

multiple ethnicities 

2-part 

14,330 (35.5) 

2,219 (5.5) 

3-part 

14,090 (34.9) 

3,308 (8.2) 

T 1 

i uiai 

oq /ion i~/n a\ 

R COT /I Q ~7\ 

vJ, / \ 1 O . / / 
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Results 

• 77.4% of test-takers responded to the special 
study questions and the SAT Q (results based on 
this sample only). 

• 92.2% of these test-takers indicated the same ethnicity 
in the special study and on the SAT Q. 

• 13.5% of these test-takers moved from a single 
ethnic/racial category to multiple. 

• If we look at the entire sample multi-ethnic/racial 
students make up 1 1 .8%. 
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What did students who are multi-ethnic 
report as their ethnicity in the SAT Q? 



N 

% 

No Response 

5,365 

9.4 

American Indian 

1,178 

2.1 

Asian American 

7,417 

12.9 

African American 

6,996 

12.2 

Mexican 

5,839 

10.2 

Puerto Rican 

1,775 

3.1 

Other Hispanic/Latino 

5,934 

10.3 

White 

16,891 

29.5 

Other 

5,957 

10.4 

Total 

57,352 

100 
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Impact to Proposed Reporting 
Categories for Ethnicity 


American Indian 
Asian 
Black 
Hispanic 
White 

Other 

Multi-ethnic 
(not Hispanic) 


] 0.6 
0.2 







_| 1 3.8 

1 1 1 .8 


3 







1 6.4 


9.9 







1 1 2.0 








_| 63.8 


39.2 


2 






1 3.7 

□ 2.1 

1 8. 


□ SAT Q 

□ Proposed 


0 10 20 30 40 50 60 70 
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Conclusions 

• A similar study conducted in 1998 indicated that roughly 
4.5% of students were multi-ethnic.* This current study 
suggests that this group may have tripled in 7 years. 

• It is clear that the U.S. population is shifting and current 
data collection and reporting practices do not accurately 
capture this population. 

• Collecting multi-ethnic data, however, can present a 
challenge. 

• Students may not respond, or respond inappropriately (checking 
all boxes) 

• It is unclear how data on these students should be reported or 
studied 


*Wendler, Feigenbaum and Escandon (2001). 
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Future Research 

• Explore ways to match students as they come to 
the website to answer the questionnaire a little at 
a time 

• Collect multi-ethnic/multi-racial in "pilot" format 
for an extended period of time (1-2 yrs) in order 
to analyze in further detail 

• Identify ways to capture multi-ethnic/multi-racial 
data without compromising trend data 
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Questions, Comments, Suggestions 

• Researchers are encouraged to freely express their 
professional judgment. Therefore, points of view or 
opinions stated in College Board presentations do not 
necessarily represent official College Board position or 
policy. 

• Please forward any questions, comments, and 
suggestions to: 

- esawtell@collegeboard.org, 

- vsathy@collegeboard.org, 

- dskrobela@collegeboard.org 
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